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PERFORMANCE AUDITS 



Performance audits conducted by the Legislative Audit Division are designed to assess state 
government operations. From the audit work, a determination is made as to whether agencies and 
programs are accomplishing their purposes, and whether they can do so with greater efficiency 
and economy. The audit work is conducted in accordance with audit standards set forth by the 
United States Government Accountability Office. 

Members of the performance audit staff hold degrees in disciplines appropriate to the audit 
process. Areas of expertise include business and public administration, mathematics, statistics, 
economics, political science, criminal justice, computer science, education, and biology. 

Performance audits are performed at the request of the Legislative Audit Committee which is a 
bicameral and bipartisan standing committee of the Montana Legislature. The committee consists 
of six members of the Senate and six members of the House of Representatives. 
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Report Summary 



Introduction 



The Child and Family Services Division (division) provides in-home 
services to children and families to mitigate actual or potential risks 
of child abuse and neglect. In-home services refer to a broad array 
of services such as teaching parents basic life-skills, coordinating 
community-based services, and providing "hard" services such as 
basic necessities for children. The division provides these services to 
mitigate risks of abuse and neglect so children may remain in their 
homes while the division works to address the underlying problems 
and to reunify families if children are removed from their homes for 
their safety and well-being. 



The Division Has Been 
Strengthening Its 
Contract Monitoring 
Practices 



The Division Could 
Expand Contract 
Monitoring 



The division relies primarily upon contractors to provide in-home 
services. Since the basic contract services procurement goal is to 
obtain the right services at the right price from the right provider, our 
main objective was to determine whether the division's contract 
management activities provide assurances contracted services meet 
contract requirements and division expectations. 

During the audit, the division began strengthening its contract 
monitoring practices, improving contractor reporting and 
management information systems, implementing onsite contract 
monitoring, and analyzing contracted service costs. Audit work 
indicated the division should continue to improve its in-home 
services contracting practices to increase contractor accountability 
and improve evaluation of contracted services. 

The division has focused contract monitoring efforts primarily on 
tracking the number of children and families receiving services, but 
does not have a full range of performance measures that provide 
assurances contractor services meet contract requirements, achieve 
expected outcomes, or that costs are reasonable. For example, 
analysis of a sample of fiscal year 2005 contracts indicated 
contractors' hourly rates ranged from approximately $31 per hour to 
$165 per hour, which suggested factors other than regional and local 
cost differences affected hourly rates. Further analysis indicated the 
division might have avoided up to $21 1,000 of contract expenditures 
in seven of the 2005 contracts by setting limits on the maximum 
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The Division Should 
Clarify Contract 
Requirements 



hourly rate. According to the division, subsequent review of 
contracts indicated improper documentation or travel requirements 
contributed to higher hourly rates. While actual cost avoidance may 
be less, without effective monitoring the division has limited 
assurance services costs are reasonable. We recommend the division 
expand in-home services contract monitoring efforts to analyze 
additional performance measures. 

The division should ensure all contracts set clear contract 
specifications. While some contracts included detailed descriptions 
of contracted services, other contracts included only general 
descriptions of service requirements or expectations. For example, 
the division expects contractors to reasonably accommodate client 
schedules by providing services during evening or occasional 
weekend hours but does not specify minimum contractor hours for 
providing services. Also, most contracts we reviewed paid 
contractors based on their operating costs, not on services actually 
provided or on contractor performance. 



The Division Can Improve 
Its Management 
Information Capabilities 



During the audit, the division stated it had modified several contracts 
to pay contractors using a fee -for- service payment schedule that will 
more clearly set standards for contractor payments. We recommend 
the division continue to improve contracts to ensure contract 
specifications are clearly stated and specify measurable criteria for 
contractor payments. 

The division could improve its management information system for 
tracking and monitoring contractor activities more efficiently and 
effectively. The existing system does not have controls to ensure 
contractors record activities uniformly and consistently. These 
limitations require additional staff time to "clean" contractor- 
reported data and compile data for reporting purposes. Additionally, 
the system was not designed for reporting purposes and requires staff 
to create and generate reports on an ad hoc basis. A desktop 
management information system (e.g.. Access or dBase) would 
increase staff efficiencies with improved data management and 
reporting capabilities. 
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The division converted from a paper-based reporting system 
to an electronic system, which improved its management 
information capabilities. We recommend the division 
continue to update its management information system to 
improve data accuracy and reporting capabilities. 
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Introduction 



Audit Scope, Objective, 
and Methodologies 



The Legislative Audit Committee requested a performance audit of 
the In-Home Services Program within the Child and Family Services 
Division (division) at the Department of Public Health and Human 
Services (department). The division provides in-home services to 
children and families to mitigate actual or potential risks of child 
abuse and neglect. The division also provides these services to 
promote reunification of families when a child has been removed 
from the home for safety reasons. 

Since the division relies upon contractors to provide in-home 
services, our main objective was to determine whether the division's 
contract management activities assure contractor accountability for 
services provided. 

To address the objective, we: 

► Interviewed division management and personnel. 

► Reviewed a sample of in-home services contracts. 

► Analyzed data from a sample of contractors' monthly activity 
reports. 

► Examined division practices for monitoring and evaluating 
contractor services. 

► Reviewed other states' contracting practices. 

Audit scope was limited to examining administration of in-home 
services contracts for fiscal year 2005. 

Audit planning work indicated the division could improve its 
contracting practices to provide better assurance contracted in-home 
services meet children and families' needs and that services costs are 
reasonable. Consequently, we focused audit scope on division 
monitoring practices. We did not examine the following contract 
administration activities: 

► Contractor selection and award processes. 

► Expenditures of program funds for program services the division 
provides. 
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► Examination of onsite contractor records. 

► Division compliance with state procurement laws. 

► Division compliance with state and federal program 
requirements. 
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Introduction 



When the division determines children are at risk of abuse and 
neglect, or have been abused and neglected, it may provide in-home 
services to: 

► Mitigate the risk so a child or children may remain in the home 
while the division works to address the underlying problems that 
pose risks. 

► Reunify the family if a child or children must be removed from 
the home for their safety and well being. 

► Promote stability and permanency for children in foster care and 
through adoptive services. 



In-Home Services Critical 
to Division Goals and 
Objectives 



The chapter presents information about the in-home services 
program. 

Federal and state laws set permanency and safety requirements for 
child protective services, which include making reasonable efforts to 
prevent the removal of children from their homes and reunifying 
families when possible. In-home services are critical components to 
the division's family preservation and child safety goals and 
objectives for complying with these laws. 



In-Home Services Refers to 
a Broad Array of Services 
and Programming 



In-home services refers to a broad array of services for children and 
their parents. In-home services can range from teaching parents 
basic life-skills to coordinating various community-based services 
with social workers and other professionals. Examples of in-home 
services include: 



► Family-based services . These services are provided in the home 
to teach parents basic life skills such as parenting, nutrition and 
food preparation, budgeting, and personal hygiene. 

► Supervised visitation . When children must be removed from 
their home, the division provides supervised visitation to help 
maintain family relationships. This service may range from 
observing visitations solely to ensure child safety to teaching 
parenting skills. 

► Case management services . These services include coordinating 
division and community -based services and resources for clients 
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and caseworkers, documenting and reporting ctient activities, and 
assessing client progress. 

► "Hard" services . Hard services include assisting families to meet 
or obtain essential needs. Hard services can include providing 
necessities, such as a crib for new mothers, or arranging 
transportation for family members to obtain services or attend 
scheduled appointments. 

The division selects contractors based on responses to requests for 
proposal. The division contracts with approximately 
15 not-for-profit organizations to provide in-home services in most 
areas of the state. The number of contractors varies depending on 
contractor availability and whether the division or contractors cancel 
or do not renew contracts. 



Program Funding 



The division spends approximately $2 million annually for in-home 
services from the state General Fund and the federal Safe and Stable 
Families grant program (IV-B program). Table 1 provides program 
expenditure information for fiscal years 2002 through 2005. 





Table 1 
In-Home Services Program Expenditures by Funding Source 






Fiscal Years 2002-2005 






Funding Source 


FY2002 


FY2003 


FY2004 


FY2005 




General Fund 


$1,457,513 


$1,022,484 


$1,164,793 


$947,705 


Federal IV-B 


707,311 


574,197 


1,022,314 


1,243,418 


Total Expenditures 


$2,164,824 


$1,596,681 


$2,187,107 


$2,191,123 


Source: 


Compiled by the Legislative Audit Division from SABHRS 





The division allocates most program funds to the division's five 
administrative regions. Funds are allocated based on factors such as 
the number of children in care and the number of abuse and neglect 
allegations. In fiscal year 2005 the division allocated 93 percent of 
program funds to the regions and retained 7 percent of the funds for 
statewide needs such as providing services to foster and adoptive 
families. Regions used approximately 75 percent of program funds 
for contracted services and remaining funds for other program costs. 
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Program Operations The division has one FTE, a program officer, assigned to manage the 

program. The program officer's responsibilities include assisting 
regional personnel with requests for proposals and contract awards, 
compiling and reviewing contractor reports, reviewing contractor 
invoices, in-home services policy development and training, and 
providing management with program information. 

Division personnel said obtaining contracted services is difficult in 
some areas. The division has authorized one region to hire personnel 
to provide these services and is considering hiring personnel in other 
regions. 
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Chapter III - Improving Contractor Monitoring 



Introduction 



Essential Contract 

Management 

Responsibilities 



This chapter presents more detailed information about the division's 
in-home services program and recommendations for improving 
management of in-home services contracting and program 
accountability. 

The basic contract services procurement goal is to obtain the right 
services at the right price from the right provider. To meet that basic 
goal, best contract management practices recommend implementing 
controls that provide assurance: 



► Service costs are reasonable. 

► Contractors provide the expected services. 

► Services meet expected standards. 

Good contract management provides greater assurance that service 
costs are reasonable and result in outcomes that help the division 
meet it's goals and objectives for children and families in the child 
protective services system (CPS). A good contract management 
system can increase staff efficiencies and provide greater personnel 
and fiscal resource flexibility. 



Monitoring Should Assure 
Services Meet Contract 
Specifications and 
Expectations 



Monitoring is essential for ensuring contractors' services meet 
contract specifications and division expectations. Without effective 
monitoring, the division has increased risks that contracted services 
do not meet division requirements or expectations such as: 



► Failing to make timely contact with clients. 

► Not providing services in client homes. 

► Reporting nonbillable hours for payment. 

► Recording program services incorrectly. 

► Not providing services as requested. 

► Not submitting timely closure reports or evaluations. 

► Services not provided by qualified staff or as specified in the 
contract. 
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Monitoring Could Be 
Expanded to Include 
Examining Costs of 
Services 



The division has been strengthening its contract monitoring 
practices. Several years ago the division began requiring contractors 
to submit monthly activity reports about individual clients using an 
electronic spreadsheet. Division personnel also began examining 
contractor costs and conducting onsite reviews of contractor 
activities and records during the course of this audit. 

Although the division monitors contract expenditures to ensure 
contractor payments do not exceed contract amounts, it has not 
examined in-home services costs related to services provided. 
Analysis of 7 of 1 6 contracts and contractor-reported information for 
fiscal year 2005 indicates costs for similar services varied 
significantly (e.g., cost per hour or cost per family). For example, 
contractors' hourly rates ranged from approximately $31 per hour to 
$165 per hour. Table 2 provides more detailed information about 
costs for contractors in our sample. 







Table 2 

Service Costs for Sampled Contractors 
Fiscal Year 2005 Contracts 








Contract 
Payments 


Service 
Hours 


Effective 

Hourly 

Rate 


Families 
Served 


Cost per 
Family 


Children 
Served 


Cost per 
Child 




Contractor A 


$ 396,545 


12,776 


$ 31.04 


313 


$ 1,267 


445 


$ 891 


Contractor B 


60,642 


1,615 


37.55 


38 


1,596 


38 


1,596 


Contractor C 


150,573 


3,093 


48.68 


115 


1,309 


124 


1,214 


Contractor D 


33,853 


446 


75.90 


20 


1,693 


51 


664 


Contractor E 


83,792 


1,028 


81.51 


46 


1,822 


118 


710 


Contractor F 


75,151 


656 


114.56 


33 


2,277 


43 


1,748 


Contractor G 


180,124 


1,090 


165.25 


36 


5,003 


85 


2,119 


Total/Unit Costs 


$ 980,680 


20,704 


$ 47.37 


601 


$ 1,632 


904 


$ 1,085 


So 


urce: Compiled 


by the Legislati 


ve Audit Division from Child am 


i Family Services Division recon 


is. 



We anticipated these costs would vary because of differences in local 
and regional operating costs such as employee wages, building costs, 
and transportation. However, the wide variations suggest other 
factors are impacting costs. Identifying these cost variations for 
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further examination should be part of a control system to help 
establish the right services at the right price. According to division 
management, their subsequent review of some contractor activities 
indicated improper contractor reporting practices and travel 
requirements impacted hourly rates for at least two contractors. 

Contract monitoring also is a useful strategy for controlling future 
costs. For example, our analysis indicated the division could have 
reduced future contract costs by setting maximum allowable hourly 
rates. Table 3 provides information on potential cost avoidance 
based on the assumption services would be available at the rates used 
in our analysis. While actual cost avoidance may be less, without 
effective monitoring the division has limited assurance service costs 
are reasonable. 
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Table 3 

Cost Avoidance Analysis for Contractor Sample 
Fiscal Year 2005 Contracts 





Actual 


Estimated at $50 per 
Hour Maximum ' 


Estimated at $65 per 
Hour Maximum ^ 


Contract 
Expenditures 


$ 980,679 


$ 768,763 


$ 817,064 


Cost Savings 


$0 


$211,916 


$ 163,615 


Cost Per Child 


$ 1,085 


$850 


$904 


Cost Per Family 


$ 1,632 


$ 1,279 


$ 1,360 


Hours of Service 


20,704 


24,942 
(4,238 more hours) 


23,221 
(2,517 more hours) 


Children Served 


904 


1,153 
(249 more children) 


1,128 
(224 more children) 


Families Served 


601 


767 
(166 more families) 


721 
(120 more families) 


Average contracted services hourly rate for contractor sample. 
^ Median contracted services hourly rate for contractor sample. 

Note: Actual cost avoidance could be higher since some contractor costs were less than 
the average and median hourly rates. 



Source: Compiled by the Legislative Audit Division from Child and Family Services Division 
Records. 



As Table 3 illustrates, the division might have avoided up to 
$211,000 in actual expenditures. By avoiding costs, the division 
could provide the same level of services with less General Fund 
money or serve more families at the same funding level. 



Effectiveness of 
Contracted Services Also 
Not Known 
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Service cost is only one performance measure. Contract monitoring 
should also include a performance measurement component that 
provides for comprehensive analysis of contracted services 
effectiveness, or outcomes. The division does not have a full range 
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of performance measures to determine the extent that contractors' 
services help the division achieve its goals and objectives. 



There are four basic types of performance measures. Table 4 
provides descriptions of performance measures and examples of 
potential in-home services performance measures. 



Table 4 
Types of Performance Measures 



Performance Measure 


Description 


Examples 


Output Measure 


A quantifiable indicator of the number of 
goods or services an agency produces. 


1 . Number of children served. 

2. Number of families served. 

3. Service hours provided. 


Efficiency Measure 


A quantifiable indicator of productivity 
expressed in unit costs, units of time, or 
other ratio-based units. 


1. Cost per hour of service. 

2. Cost per child served. 

3. Cost per family served. 


Outcome Measure 


A quantifiable indicator of the pubUc and 
customer benefits from an agency's 
actions. 


1 . Rate of substantiated abuse 
and neglect incidents. 

2. Rate of children re-entering 
the child protective services 
system. 

3 . Rate of children re-entering 
foster care. 


Explanatory Or Input 
Measure 


An indicator of factors, agency resources, 
or requests received that affect 
performance. 


1 . Number of children or families 
referred. 

2. Child or family risk 
assessment. 

3. Reasons families exit in-home 
services programming. 



Source: Compiled by the Legislative Audit Division. 



The Division Could 
Expand Contractor 
Monitoring to Include 
Additional Performance 
Measurement 



The division could expand contractor monitoring to include 
additional types of performance measures. While the division has 
tracked the number of children and families served and recently 
started monitoring service costs, it has not established outcome 
measures or identified explanatory measures for more extensive 
analysis of contractor services. With its improved contractor 
reporting, the division already collects some information necessary 
for additional analysis. The division should continue to develop 
additional relevant performance measures. The division can 
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subsequently modify performance measures and data collection 
efforts as it refines its evaluation needs. 



The Division Should 
Clarify Contract 
Expectations and 
Requirements 



Recommendation #1 



We recommend the Child and Family Services Division expand 
in-home services contractor monitoring efforts to analyze 
additional outcome and explanatory performance measures. 



The division should also modify its contracts to better define and 
clarify contractor performance expectations. Some contracts 
provided only general descriptions of services to be provided. Other 
contracts, however, provided specific details about provided 
services. For example, one contract included weekly schedules of 
contractor activities with clients. Another contract included 
descriptions of how services are presented to clients, such as 
presenting parenting information, modeling good parenting skills, 
and evaluating client-parenting skills. The following examples 
illustrate the types of problems that can occur when contract 
requirements are not specific. 



Contract Payments Not 
Based on Services Provided 



► Contractor availability . The division expects contractors to 
reasonably accommodate client schedules by providing services 
during evening or occasional weekend hours. Contracts do not 
specify minimum contractor hours for providing services. 

► Billable hours . Fee-for-service contracts do not define "billable" 
hours. For example, contracts do not specify whether 
contractors may bill the division for time spent writing reports 
and preparing services plans or only for direct contact time with 
clients. 

Most in-home services contract payments are based on contractor 
operating costs, not services provided or contractor performance. 
In-home services contracts include contractors' annual budgets for 
providing services to a specified number of families. Annual 
budgets are based on estimated operating costs, such as wages, 
equipment, and utilities. For most contracts, contractors submit 
monthly invoices of their operating costs for payment. 
Consequently, the division reimburses contractors for their costs 
regardless of services provided. 
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other Agencies' Contracts 
Include Clear Contract 
Specifications 



Standard contracting practices and state policy require clear contract 
specifications to ensure contractor accountability. Examples of 
better contract specifications we identified in contracts and requests 
for proposals (RFPs) from other state and local child protective 
services agencies included: 



Contract Requirements 
Based on Contractor 
Proposals 



► Minimum standards for providing crisis intervention services. 

► Minimum standards for client outcomes, such as percentage of 
clients demonstrating improved parenting skills. 

► Definitions of billable services or billable time for individuals 
and family units. 

► Specific requirements for evaluation and assessment services, 
including staff qualifications. 



In an effort to streamline the contracting process, the division uses 
generic language and incorporates, by reference, other contracting 
documents such as the RFPs and contractor services proposals. 
However, RFPs do not include performance measures or clear 
specifications for some requested services, and the detail in 
contractor proposals can vary significantly. Some contract 
requirements refer to division in-home services policies, which may 
describe what a contractor "should" do or what is "anticipated," but 
not what is required for payment. 



Without clear specifications, the division has less ability to hold 
contractors accountable for services provided and increases risks it 
will pay contractors for services that do not meet minimum 
expectations or for unexpected costs. Division management also 
stated additional time is spent responding to misunderstandings or 
disagreements resulting from unclear specifications. 

During the audit, the division management stated it had modified 
several contracts to pay contractors on a fee-for-service basis. The 
division should continue its efforts to improve its contracting process 
by ensuring specifications for in-home services contracts are clearly 
stated and provide better assurance contractors are accountable for 
provided services. The division could set minimum specifications 
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and expectations in RFPs, and develop standardized contract 
sections specific to in-home services. 



Recommendation #2 



We recommend the Child and Family Services Division ensure 
that in -home services contracts: 

A. Define minimum specifications and expectations. 

B. Specify measurable criteria for contractor payments. 



The Division Could 
Improve its Management 
Information Capabilities 



The division could improve its management information system for 
tracking and monitoring contractor and program information. Our 
analysis identified two primary areas for improvement. 

► Data collection and reporting controls . The electronic reporting 
spreadsheet used by contractors to record activities does not have 
system edits to ensure data is entered uniformly and consistently. 
Inconsistencies among contractors ranged from using different 
terms for documenting services to modifying client identification 
numbers. 

► Data analvsis and reporting . Contractor monthly reports are not 
maintained as a program database, but in separate files and 
worksheets that must be manually combined before using the 
data for management and program reporting purposes. 
Additionally, the system is not designed for data analysis and 
reporting purposes. 

A good management information system should have several basic 
qualities: 

► Provide quality data through internal controls that minimize data 
entry errors. 

► Provide ready and timely access to data. 

► Provide accurate and descriptive management information and 
reports that support performance evaluation. 



Existing System is 
Time-Consuming 



The current system's design is time-consuming, requiring significant 
staff time to prepare data for reporting purposes. Program staff must 
first edit or "clean" contractor data. Since data is maintained in a 
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fragmented information system, program staff must also compile 
monthly reports for each contractor before monthly, quarterly, or 
annual data can be analyzed. The program officer estimated she 
spends approximately 280 hours annually cleaning and compiling 
data. More analysis of management information could be completed 
if less staff time was required for these tasks. 



System Has Limited 
Reporting Capabilities 



The system was not designed for reporting purposes and requires 
staff to create and generate reports for each contractor or aggregate 
data, a time-consuming process. The division significantly improved 
its management information several years ago when it transitioned 
from paper reports to electronic spreadsheet reports. During this 
transition, the division also required contractors to report better 
information about services and clients served. The division has used 
the information to generate basic reports such as the number of 
clients served, minutes of services provided, and during the audit 
began using information to examine costs of services. 



A Desktop Management 
Information System Would 
Increase Staff Efficiencies 



Using a desktop management information system (e.g.. Access, 
dBase, or Excel) would increase staff efficiencies for minimal costs. 
Information technology professionals estimated developing a system 
would require approximately 40 to 60 hours of programming and 
staff time. Some additional resources would likely be needed to 
provide training to contractors and division staff, which could 
potentially be included in training the division provides to 
contractors and division personnel. In return, a desktop management 
information system would: 

► Improve data quality and reduce time necessary for cleaning and 
compiling contractor reports. 

► Organize data for timely and ready access to program 
information. 

► Provide automated reporting capabilities. 

► Increase contract and program monitoring capabilities. 

► Increase capabilities for identifying contractor and program risk 
and targeting monitoring activities. 
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The division could potentially save more than 240 hours annually, 
increasing staff time for onsite reviews and other contracting 
activities. During the audit, division management stated it would be 
developing and implementing a desktop management information 
system and training staff on its use and capabilities. 



Recommendation #3 



We recommend the Child and Family Services Division 
continue to update the in-home services management 
information system by strengthening: 

A. System edits to address data accuracy and uniformity. 

B. Data analysis and reporting. 
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DEPARTMENT OF 
PUBLIC HEALTH AND HUMAN SERVICES 




BRIAN SCHWEITZER 
GOVERNOR 


JOAN MILES 
DIRECTOR 
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RECEIVED 

MAY 3 2006 
LEGISLATIVE AUDIT OIV. 



May 30, 2006 

Mr. Scott A. Seacat 

Legislative Auditor 

Office of the Legislative Auditor 

State Capitol, Room 160 

Helena, Montana 59620-1705 

Dear Mr. Seacat : 



The Department of Public Health and Human Services appreciates the 
Legislative Audit Division's insightful recommendations resulting from 
its Performance Audit of In-Home Services (04P-12). 

Although Child and Family Services doesn't agree with all statements 
made within the audit report we concur with its three recommendations 
for improving In-Home Services contract monitoring and will integrate 
these recommendations. A discussion of Child and Family Services' 
perspective regarding its major recommendations and corrective action 
plan is attached. 



Please contact me if you have questions. 



Sincerely, 



(^IoaJ /IuU 



12^ 



Joan Miles, 
Director 



Cc Bruce Deitle 

Shirley K. Brown 
Marie Matthews 
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Improving In-Home Services Contract Monitoring Performance Audit 

Response and Corrective Action Plan 

05/30/06 



Recommendation #1: Child and Family Services Division expand In-Home Services 
contractor monitoring effort to analyze additional outcome and explanatory performance 
measures . 

Concur 

Over the past two and a half years Child and Family Services has developed and 
implemented a new Excel reporting system for hi-Home Services contracts. This 
implementation was a significant step in quantifying its contract evaluation and producing 
more specific reports. This system allowed the audit to examine data elements that would 
otherwise not have been available. However, because the system was so new, the audit 
had to rely upon data from the second year of this system's operation; therefore, some data 
elements are best viewed as preliminary since the process of data collection was and is still 
being refined. 

Some of the on-site reviews conducted recently pointed out that contractors were not 
accurately reporting service delivery units which inflated the dollar amount contained in 
the audit report. CFSD site visits identified reasons underlying some of the high hourly 
rates, such as underreporting of certain activities and the need for extensive travel in some 
rural areas. Corrective action has already been taken at sites where service delivery 
problems, reporting problems or high hourly rates were noted. CFSD will conduct 
ongoing site visits to evaluate contractor performance after implementation of fee-for- 
service provisions. 

CFSD agrees that it would be advisable to look at both outcome and exploratory measures 
more comprehensively. The In-Home Services program has been producing outcome data 
and has begun producing cost effectiveness measures. Taking this to another level, CFSD 
acknowledges some limitations with existing performance measures. CFSD's current 
outcome indicators are limited to families currently receiving In-Home Services within 
specified time fi-ames. For example, the number of out-of-home placements, placements 
prevented, and transitions from one type of placement to another (such as from non-related 
foster home to a kinship foster home). 

Regarding exploratory performance measures, CFSD currently collects case closure dates 
and the reason for closure. However, before making calculations in this area, additional 
contractor training is needed to provide clarity in defining case closure. Other exploratory 
measures are not possible within the current system. This would require data connection 
between different systems to look more comprehensively at causal patterns over time. 
CFSD can manually calculate certain outcome measures, but ideally this could be 
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accomplished electronically, if the cost is allowable. CFSD will explore these options and 
associated costs. 



Recommendation #2: Child and Family Services Division ensure that In-Home Services 
contracts: A. Define minimum specifications and expectations B. Specify measurable 
criteria for contractor payments 

Concur 

CFSD has implemented changes in the In-Home Services billing method with some 
contractors, moving from monthly cost reimbursements to fee-for-service, paying only for 
direct service hours. The remainder of In-Home Services contractors currently bill for 
their costs. CFSD has informed all current providers that upcoming contracts will require 
the use of a fee-for-service billing method. This will allow for measurable criteria upon 
which to base payments (service hours provided). CFSD has advocated for 
implementation but has struggled with service issues in rural areas, as well as identifying a 
workable and appropriate definition of billable hours that can be applied statewide by all 
contractors. A definition of billable hours, in keeping with the audit recommendation, will 
be incorporated into the In-Home Services contracts. 

CFSD currently uses boilerplate contracts with program specifics listed in attachments; this 
allows for efficiency and continuity where needed across all types of contracts the Division 
holds. Where possible a set of statements that apply to a set of contracts will be moved 
from the attachments into the body of the contract. For example information currently 
contained in the "scope" and CFS "policies and procedures" will be incorporated to help 
underscore the importance of key requirements and promote consistency in service 
delivery practices among contractors. 

To maintain contract flexibility not all program specifics will be moved into the body of 
the contract. Contractors provide a wide array of services through individualized Family 
Service Plans, thus contract language needs to be flexible to meet families' needs. Service 
hours will be defined by funding amount and rate, however requiring a set configuration of 
services with corresponding measurements for all families' services would result in 
"canned" Family Service Plans incongruous with a families' needs. 



Recommendation #3: Child and Family Services Division continue to update the In-Home 
Services management information system by strengthening: A. System edits to address 
data accuracy and uniformity, B. Data analysis and reporting. 

Concur 
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CFSD agrees that the current In-Home Services information system should be 
strengthened. This would allow the program manager more time to perform monitoring 
tasks. CFSD has contacted a staff person in Eastern Montana to provide in-house technical 
assistance to convert the current Excel system to an Access system. It is anticipated this 
conversion will reduce the number of hours staff currently spends merging and 
reconfiguring spreadsheet data. The insertion of system edits is an appreciated 
recommendation, which likewise will reduce staff time spent correcting contractor data. 

The program manager has also requested and received software programs that may serve to 
expedite data analysis. Once these programs are reviewed, including the cost, CFS will 
know whether any are a viable option. 

CFSD proposes adopting the audit recommendations as follows. 



PLANS FOR IMPLEMENTATION 



Recommendation #1: 


Completed By: 


Continue to conduct formal contract 
monitoring visits (one site per month) 


October 1,2007 


Provide training on problem areas 
(inconsistencies, coding, definitions, 
omissions) in reporting system, and/or 
incorporation of new elements to be 
measured. In conjunction with regional 
policy Training in September 2006 — five 
training sites. 


October 1,2006 


Recommendation #2: 




Conversion of most, if not all, contracts to 
a fee-for-service billing method 


July 1,2006 


Incorporate general types of 
specifications into contract 


July 1,2006 


Recommendation #3: 




Explore costs and systems 


September 1,2006 


Train contractors on all reporting changes 
in conjunction with regional policy 
Training in September 2006 — five 
training sites. 


October 1,2006 


Begin implementation of improved data 
management system 


October 1,2006 
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